Segmentation is a crucial task in visualization and quantification based on digital medical image data. The number of digitally stored medical images and the amount of information stored in each image both are steadily increasing, thus increasing the importance of robust and fast segmentation methods at the same time. Robustness refers to the ability of the method to: (i) work in presence of image disturbance, (ii) with a low failure rate, and (iii) to work for a wide range of different images and patients.
Still many specific segmentation problems persist that have not been solved satisfactorily. Each problem is defined by: (i) the imaging modality, (ii) the imaged organ, and (iii) the task to be performed on the images. Examples for such segmentation problems are removal of non-brain tissue in three-dimensional (3D) magnetic resonance (MR) images of the human head, or segmentation of left and right ventricle in cardiac MR cine images on all slices and time points.
The prior art shows a variety of segmentation methods:
Manual Segmentation
Manual tracing for object segmentation is usually performed on all two-dimensional slices of a three-dimensional image dataset that contain the structure of interest. Manual interaction is preferably applied on an intermediate level of detail, where local and global image features have to be taken into account. In these cases, manual interaction often yields more accurate results than fully automated methods.
For example, when tracing the brain contours on one 2D slice that is part of series of MR images, a trained operator combines detailed knowledge about the individual anatomy (such as brain, cerebellum, peripheral nerves and neighboring blood vessels).
Segmentation problems of this kind are suited for manual processing. Even within the framework of semi-automated methods for the, e.g., histogram based quantification of different tissue types, manual tracing is the preferred method for the selection of an appropriate region of interest (ROI).
U.S. Pat. No. 5,262,945 shows such a method for quantification of brain volume from magnetic resonance images.
There are several problems with manual segmentation preventing it from being routinely used in many cases: (i) Interaction and time costs (manual tracing takes several minutes per image), (ii) requirement for sufficient training (the segmentation result is highly dependent on the level of training), (iii) requirement for care and concentration during segmentation (quality and precision of human interaction is not constant), and (iv) reduction of objectivity (different operators will generate different results).
The growing number of digitally stored medical images as well as the advances in scanner resolution, combined with high personnel costs are another reason for manual segmentation to become even less applicable in the future. At the same time, the need for efficient and reliable automated or semi-automated segmentation methods is growing.
Thresholding and Morphology
The second group of segmentation methods is thresholding in combination with morphology, as described in K. H. Hoehne and W. A. Hanson, “Interactive 3D segmentation of MRI and CT volumes using morphological operations”, J Computer Assisted Tomography 16(2): 185–294, 1992.
As a first step, the structure of interest is roughly separated from the background by simple gray scale thresholding, resulting in a binary image. In the brain segmentation problem on T1-weighted MR images, very bright (e.g. fat) and very dark (e.g. bone, air) parts of the image are suppressed using appropriate thresholds. In some cases, the brain is represented by the largest remaining connected structure. This structure in the binary image can be applied as a mask to the original data.
In most cases, however, the structure representing the brain in the binary image is connected to neighboring structures (caused by nerves, blood vessels, image non-uniformity, or limitations of image resolution). In order to separate the brain from other structures, morphological filter operations can be used; morphological opening suppresses thin structures whilst preserving thicker ones. L. Lemieux, G. Hagemann, K. Krakow, and F. G. Woermann, “Fast, accurate, and reproducible automatic segmentation of the brain in T1-weighted volume MRI data”, Magnetic Resonance in Medicine 42(1): 127–35, July 1999, describe a series of thresholding and morphological operations that aims to solve the shortcomings of this simple approach.
Another application of thresholding is described in U.S. Pat. No. 5,903,664. For fast segmentation of cardiac images, a ROI and a seed point within the ROI are manually selected. An operator-defined threshold is used to acquire a first segmentation of, e.g., the left cardiac ventricle on that slice. This segmentation is defined as a set of connected pixels showing gray values above the threshold and including the seed point. From that first segmentation, a histogram is computed. A further threshold is selected which separates modes of the histogram and is used to carry out a revised, final segmentation of the current image. The centroid of the current segmentation is used as a seed point in segmenting adjacent images.
Within the framework of thresholding and morphological operations, the user normally interactively specifies thresholds, filter sizes, or the number of iterations, in order to produce a correct segmentation. A fully automated system of thresholding and morphology tends to fail in many cases, e.g., when the image acquisition parameters vary. Furthermore, the results often need final manual editing.
The results presented by Lemieux et al. (1999) show a progress in terms of automated thresholding and morphological operations. However, this method is tuned to a narrow range of image types. In particular, the result is dependent on the overall homogeneity of the image. The algorithm will fail in presence of moderate contrast non-uniformity, since no threshold can be found that is appropriate for all parts of the image.
In general, this weakness against image inhomogeneities is characteristic for all segmentation methods comprising global thresholding.
Statistical Methods
U.S. Pat. No. 5,425,368 describes a method for estimating tissue volumes in MR images. More precisely, a fuzzy logic approach to the problem of distinguishing cerebrospinal fluid, gray and white matter pixels is presented. An unsupervised fuzzy clustering procedure based on a variant of the fuzzy c-means algorithm computes the percentage of each of these three compartments in each image automatically, with virtually no operator intervention. Each volume element represented in the image can belong to all compartments in varying degrees. The procedure requires input of the number of different compartments in the image, as well as a parameter which determines the amount of overlap of compartment boundaries. The method has been applied to the diagnosis of hydrocephalus.
Statistical segmentation methods are useful, but target another class of segmentation problems. In general, statistical methods only converge to an appropriate solution after the original image has been segmented, so that a small set of known tissue types remains plus additional noise and non-uniformity. A statistical method could be, for example, a subsequent step to brain segmentation, also referred to as skull stripping, i.e. the removal of non-brain tissue in neuroanatomical images. Conversely, statistical methods such as described in U.S. Pat. No. 5,425,368 are not applicable to the original MR images directly.
Deformable Template Based Segmentation
The third group of segmentation methods uses deformable templates. From A. M. Dale, B. Fischl, and M. I. Sereno, “Cortical surface-based analysis I: Segmentation and surface reconstruction”, NeuroImage 9: 179–194, 1999, a surface model in terms of a tessellated mesh is known. This model is fitted to the brain surface in T1-weighted MR images in order to separate the brain from surrounding tissue. During the fitting process, two constraints are applied to the surface geometry: (i) A smoothness term (avoiding unnaturally folded surfaces), and (ii) an image term (keeping the surface close to the desired structure, here the brain surface).
During the fitting process, the surface is deformed by iteratively applying these constraints. Since the constraints take into account neighborhood information, and this information is iteratively propagated over the surface, deformable templates are suited to solve non-local segmentation problems. Deformable templates seem to be more robust and easier to automate than thresholding and morphology.
The goal of Van der Geest et al (R. J. van der Geest, V. G. M. Buller, E. Jansen, H. J. Lamb, L. H. B. Baur, E. E. van der Wall, A. de Roos, and J. H. C. Reiber, “Comparison between manual and semi-automated analysis of left ventricular volume parameters from short-axis MR images”, J Computer Assisted Tomography 21(5): 756–765, 1997) is to evaluate a semi-automated template based contour detection algorithm for the quantitative analysis of cardiovascular MRI. Left ventricular function parameters derived from automatically detected endocardial and epicedial contours were compared with results derived from manually traced contours in short-axis multislice GRE MRI studies of 10 normal volunteers and 10 infarct patients. Compared with manual image analysis, the semi-automated method resulted in smaller systematic and random differences for volume, ejection fraction, and wall thickness of the left cardiac ventricle.
A problem with template driven segmentation is caused by geometrical or anatomical variations. For pathological cases, e.g., templates often are not applicable. Interaction techniques integrated in template based segmentation algorithms can only serve to constrain parameters, but not to deal with variations beyond the model.
Segmentation Based on the Watershed Transformation (WT)
The aim of Sijbers et al. (1997) (J. Sijbers, P. Scheunders, M. Verhoye, A. Van der Linden, D. Van Dyck, and E. Raman, “Watershed-based segmentation of 3D MR data for volume quantization”, Magnetic Resonance Imaging 15(4), 1997) is the development of a semi-automatic segmentation technique for efficient and accurate volume quantization of Magnetic Resonance (MR) data. The proposed technique uses a variant of an immersion based watershed algorithm which is applied to the gradient magnitude of the MR data and which produces small volume primitives.
The known drawback of the watershed algorithm, oversegmentation, is reduced by a priori application of a 3D adaptive anisotropic diffusion filter to the MR data. Furthermore, oversegmentation is a posteriori reduced by properly merging small volume primitives which have similar gray level distributions. The outcome of the preceding image processing steps is presented to the user for manual segmentation. Through selection of volume primitives, the user quickly segments the first slice which contains the object of interest. Afterwards, the subsequent slices are automatically segmented by extrapolation.
U.S. Pat. No. 5,463,698 shows a fast implementation of the watershed transformation (WT) with markers based on hierarchically organized queues. The problem that is solved by the present invention, being the separation of structures that exhibit similar gray scale characteristics, is not touched by this realization.
Hahn et al (H. K. Hahn and H.-O. Peitgen, “The Skull Stripping Problem In MRI Solved By A Single 3D Watershed Transformation”, Medical Image Computing and Computer-Assisted Intervention (MICCAI 2000), Pittsburgh, Springer LNCS: 134–143, 2000), the disclosure of which being expressly incorporated herein by reference, have been the first to describe the direct application of the WT to a three-dimensional medical imaging problem. No gradient or edge information is derived prior to the transformation.
A priori, it is not clear that such a strategy could produce the desired results. However, there is a class of problems that can be solved by the direct application of the WT. In these cases the main problem consists of separating an object of interest from neighboring structures that exhibit a similar image intensity, but are separated by a thin layer of slightly darker or brighter tissue. In Hahn et al. (2000) the brain was separated in T1-weighted MR images, much as in Dale et al. (1999), but with an increase in robustness.
Watershed segmentation offers promising approaches to robust segmentation of images. The WT can be directly extended to 3D and even 4D and higher dimensional problems. However, the gradient information of an image, to which the WT is applied in accordance with the prior art, is only suited to yield appropriate results if the object borders exhibit image gradients that are higher than the gradients inside the object.